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Paine College, a four-year liberal arts institution 
in Augusta, Georgia, serving costly black students recently completed 
an assessment and evaluation plan aimed at all units of the college* 
The plan was developed by members of the Committee on Institutional 
Effectiveness and Research over 12 mortths and focuses on the school's 
primary mission to provide quality education. The committee chose a 
format based on responses to seven key issues: (1) what must be 
asserted? (2) who does the assessing? (3) when is assessment done? 
(A) what is the system of prioritization? (5) how do assessment 
results flow among units? (6) what provisions exist for feedback and 
use of results? and (7) how is the evaluation plan itself evaluated? 
Use of this structure allowed the plan to provide an over-arching 
framework for assessing the institution including determining 
educational and administrative outcomes and describing how the 
intended results would be achieved, evaluated^ assessed, and utilized 
for institutional improvement. The plan was implemented by the 
committee and distributed to senior and mid-level administrators. 
Deadlines were announced and workshops were held. Data were collected 
on an ongoing basis. Detailed outcome goals were submitted by 
department heads for review. In its preliminary stages the plan's 
main impact has been campuswide involvement. (JB) 
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ABSTRACT 

The Paine College Comprehensive Assessment and Evaluatim Plan (CAE? or the Plan) is the guiding 
document for the evaluation of institutional effectiveness at the college. It defines the organizational 
scheme for the evaluation of all operations and outlines strategies for assessing the extent to which the 
mission is achieved. By May 1993, all units will have developed outcome statements and 12 units of the 
college will have been evaluated based on their defined outcomes. Development of the Plan, its first year 
of operation, problems, and solutions in its implementation form the basis for the presentation. 
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The Paine College 
Comprehensive Assessment and Evaluation Plan: 

A Model for the 
Implementaticn of Institutional Effectiveness 

at a Small College 

INTRODUCTION 

Paine CoUege, a four-year liberal arts insiitution, located on 54 acres in Augusta, Georgia awards 
bachelor of arts and bachelor of science degrees in 13 majors. Seventy-seven percent of its 686 students 
are from the state of Georgia; almost three percent are foreign. Its faculty has been biracial almost since 
its founding while its student body has been and continues to be predominately black. Paine College is 
supported by the United Negro College Fund, the Christian Methodist Episcopal Church and The United 
Methodist Church. Under the auspices of the Board of Trustees, the President with four other senior line 
officers administers the affairs of the college. 

Throughout its 111-year history, Paine College has been an effective and vital institution, using its 
resources and employing appropriate strategies and processes to fulfill its nussion. The recently 
completed Paine College Comprehensive Assessment and Evaluation Plan (CAEP or the Plan), is the 
overarching framework for assessing and evaluating all units of the coUege. The purpose of this p£\per is 
to describe the Plan, summarize implementation strategies, and highlight the potential impact on the 
institution. 

The Paine College CAEP was developed by members of the Committee on Institutional 
Effectiveness and Research (OER) over a twelve-month period in response to several "must" statements 
in the Criteria and a recommendation from the Southern Association of Colleges and Schools following 
the completion of the college's Instihitional Self-Study in 1991. The CIER is chaired by the director of 
Institutional Effectiveness and Research, and is composed of chairs of each of the five academic divisions 
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in addition to representatives from educational support, student development, administrative processes, 
and the student body. 

PHILOSOPHY OF INSTITUTIONAL EFFECTIVENESS 
Paine College's philosophy of institutional effectiveness is grounded in the belief that the 
institution's primary considerations must be the quality of education that it provides as well as the 
processes through which that education is imparted. The assessment of the college's effectiveness in 
providing that education involves a systematic, explicit, and documented comparison of institutional 
performance to institutional purpose. Thus, institutional effectiveness is measured by the extent to which 
Paine College fulfills its stated mission and the goals and objectives derived from it 

Paine College is committed to an excellent instructional program and directs its academic 
activities, including faculty development, proposal writing, and workshops, to that end. The college 
strives to instill confidence, self-esteem, and pride of heritage in students. They learn that the boundaries 
of achievement ara neither defined nor prescribed by race, but by ability and perseverance. Thus, five 
postulations underlie the philosophy of effectiveness at Paine College: 

1. Paine College is a teaching institution where quality and effectiveness of instruction are 
major fod. 

2. A liberal arts education changes the whole student, and this transformation often eludes easy 
packaging, oversimplification, and measurement. 

3. There mu*'*: be a variety of indices to gauge institutional effectiveness. 

4. Outcomes directed by value-added approaches to effectiveness must be a major focus. 

5. While institutional planning is the responsibility of the Committee on Strategic Planning 
(CSP), and while a management system is an integrated approach to planning and 
evaluation, many formal and informal activities must coalesce into an effective planning 
whole. 
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THE PLAN 

After reviewing various resources on planning, management, evaluation, assessment, and 
effectiveness, members of the CIER determined that the traditional strictly narrative format was 
inappropriate for Paine's CAEP. Thus, responses to the following seven questions define the components 
of the Plan and are summarized following the list 

1 . What must be assessed? 

2. Who does the assessing? 

3. When is assessment done? 

4. What is the system of prioritization? 

5. How do assessment results flow among units? 

6. What provisions are there for feedback and use of results? 

7. How is the Comprdiensive Assessment and Evaluation Plan evaluated? 

What must be assessed? 
Both objective and subjective data is assessed* Assessment is implemented at the unit level and 
monitored by the Committee on Strategic Planning (CSP) and the Committee on Institutional 
Effectiveness and Research (QER). The head of each senior and mid-level administrative unit is 
responsible for guiding his or her members through the process of developing outcomes, whose main 
fod are defining expected results, accompanying assessment procedures, and linking them to the 
collegers purpose. Forms lEl: Educational Outcome and IE2: Administrative Outcome are used to define 
all aspects of the expected results. 

Who does the assessing? 
Insuring institutional effectiveness is the responsibility of every member of the college 
community. Thus, everyone is involved in assessment and evaluation. Leaders are members of the 
Board of Trustees, the President, all senior administrators, division chairpersons, department 
coordinators, other budget supervisors, and all program heads, directors, and supervisors. The intent is 
that those persons closest to a specific area monitor the operations in the area and make necessary 
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changes for improvement. CIER's role is to oversee the assessment process. Figure 1 graphically depicts 
the various roles. 
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Fi gure 1 Institutional effectiveness model designed to show the college-wide relationships 
among the major activities and leaders in the implen .utation of the CAEP. 

In Figure 1, implementation includes the development of educational or administrative 
outcomes. Assessment includes the identification and administration of evaluation procedures and the 
measurement of the extent to which the desired results as defined in the outcomes were obtained. 
Evaluation results are disseminated to the appropriate offices for use in planning and nuking 
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adjustments in programs and operations, and are coalesced into a formal annual report in the spring and 
a summative report every five years. This feedback of the results is indicated by arrows in the diagram. 

When is assessment done? 
The overall calendar for assessment is based on an academic year, since the primary 
consideration of institutional effectiveness is the quality of education that the college provides. Paine 
College uses two types of formal assessment-five- or ten-year summative evaluations and annual or 
otiier formative evaluations. The rotating schedule is given in the document in the section that describes 
the system of prioritization. Since this is tiie first year for the completion of sununative evaluations, the 
period to be covered spans two instead of five years. 

What is the system of prioritization? 
The system of formal evaluation contained in tiiis document is based on the need to complete 
reviews for external reporting sources while maintaining a sense of direction internally. Thus, while all 
offices or programs submit annual reports, not all are extensively evaluated each year. Components of 
academic affairs will be evaluated each year. Conceptually, all majors are evaluated prior to tiie division 
evaluation. Non-major components are evaluated during division assessments unless otherwise listed. 
Each senior level administrative area will be evaluated once during a five-year period. Other programs 
or administrative units will also be evaluated once during a five-year perio4^ Based on current 
assessment efforts, a schedule for 1992 tiirough 1997 appears in the Plan as Exhibit 2. 

How do assessn\ent results flow among units? 
There is, at least nominally, a hierarchical system for the flow of irtformation that follows a 
"chain-of-command" or "distributive" model which largely mirrors the college's organizational chart. 
What provisions are there for feedback and use of results? 
Provisions for feedback and use of results are explicitiy stated in each outcome. Thus, the system 
for feedback and use of results is pro-active. A format for reporting annual and five-year assessment 
results appears as an appendix in the Plan. 
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How is the Comprehemive Assessment and Evaluation Plan evaluated? 
This Plan is a dynamic document tliat will be evaluated annually in the fall and updated in the 
spring by members of the Committee on Institutional Effectiveness and Research (CIER). Outcomes 
created at the unit level and appended to this plan will be modified as indicated in each outcome. 
Reports detailing accomplishments at each level will be revie ved by CIER and recommendations made 
to the Committee on Strategic Planning for appropriate action and implementation. 

Sununary of the Plan 

The Comprehensive Assessment and Evaluation Plan is the overarching framework for assessing and 
evaluating all units of the college. Detailed educational and administrative outcomes describe how the 
intended results will be achieved, evaluated, assessed, and utilized to make improvements in the college's 
programs, therefore producing more effective graduates. Completed outcomes are appended to the plan 
and reports of results are filed in appropriate offices. 

IMPLEMENTATION 

Implementation has been facilitated by having the Committee on Institutional Effectiveness and 
Research meet every other week to pkn activities and review and evaluate materials. The Plan was 
distributed in a workshop format to members of the Committee on Strategic Planning, whose 
membership includes all senior and 11 mid-level administrators. Plans for the year were discussed and 
everyone's role in the implenwntation of the Plan was discussed. To facilitate the completion of the 
outcomes, "five-year" reports, and conunittee operational procedures, three additional workshops were 
held. Deadlines were announced for the completion of all information. Data is collected on an on-going 
basis with the annual Paine College Fact Book and other resources providing much of the necessary 
baseline ii\formation. 

Detailed educational and administrative outcomes have been developed for 1992-1993 by 23 of 
the original 48 heads who were to have submitted outcomes. Thirteen of the original 48 were collapsed 
into the requirements for their supervisors. Therefore, outcomes are expected from 35 individuals. 
Twelve sets or partial sets of outc jnves are still outstanding. Thirteen department or program heads for 



10 



Institutional Effectiveness Model 

9 



which outcomes had not been completed by the second deadline have been requested to complete them 
for 1993-1994 and submit them by the end of the semester. One person has complied with this request It 
is expected that others will do so in the near future. 

The Committee on Institutional Effectiveness and Research has been reviewing the outcomes 
from academic affairs since February, 1993. When blatant omissions or errors tlut suggest a lack of 
understandir^g of the content that is to be included have been noted, the outcome has been returned to U.e 
appropriate head for correction. During the summer, the review will continue. Plans are to submit 
reports summarizing the expected results and the progress toward achieving the results to the Committee 
on Strategic Planning as each administrative section (academic affairs, fiscal affairs, student services, and 

institutional advancement) is completed. 

Faculty committees were to have created detailed operational procedures following tiae approval 
of the current By-Urvs of the Faculty. Since this had not occurred, the committees were instiructed to 
complete the operational procedures during this year instead of outcomes. Four com^nittees out of 11 
have submitted operational procedures to date. All others are editing tiaeir work and plan to meet the 
April 30th deadline. All procedures will be reviewed, revised, and subsequentiy approved by the faculty 
in the fall and become appendices to the ByLam. Plans are to engage the chairs of all committees m a 
workshop on developing outcomes after the approval of the operational procedures in the fall. This will 
bring them on line for the creation of outcomes as defined in the Plan and the expected results defined in 
the outcomes will provide a basis for evaluating committee activities. 

Twelve units of the college are in the process of completing an extensive analysis of their 
operations and activities. Requests for assistance with these two-year reports have been filled. Again, the 
April 30th deadline provides sufficient leeway for the completion of tiie reports. 

IMPACT ON THE INSTITUTION 
Results to date suggests that the coUege will have sufficient data to document effectiveness or to 
make recommendations for change in aU areas. The major impact on ti^e institiition has been campus- 
wide involvement. As indicated earlier, the Paine College Comprehensive Assessment and Evaluation Plan 
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was developed by a college-wide committee. Further, outcomes are developed and assessed at the unit 
level. The whole process is fadUtated by having a president who is aware of the need to engage in 
assessment activities and who has served as a leader in the development and implementation of the Plan. 

Also worth noting is the development of a greater awareness of the need to adopt plans and 
follow through on defined activities on th6 part of unit heads in addition to an awareness that the self- 
study process has changed from a process of extensive evaluation every 10 years to an on-going activity. 
Last but not least, improvements in various areas are discussed and acted on as the need arises instead of 
waiting until an external agency has a scheduled visit or recommends changes or improvements. 

The experience of implementing the plan has been cliallenging while not being overwhelming. 
Most individuals have cooperated to the fullest extent possible. Since the deadlines that were established 
are internal, the Committee on Institutional Effectiveness and Research has been very patient with those 
who have missed them and will evaluate the effect on implementing the Plan in the fall 

FORUM HANIX)UT 

A handout containing the following items will be distributed to session participants at the 1993 

Forum. 

Presentation Transparencies 
AIR paper 

Workshop Agenda, 11/19/92: Implementing The Paine College Comprehensive Assessment and 
Evaluation Plan, Development of Educational and Administrative Outcomes with Materials 
List on back 

Blank copies of Forms lEl and IE2 

Sample 1: Educational Outcome for Philosophy and Religion 

Sample 2: Administrative Outcome for Institutional Effectiveness and Research 

Evaluation Form (Workshop, 11/19/92) 

T^: Toward Two Thousand, The Paine College Strategic Plan (EOT, 10/29/92), cover and table of 
contents 
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Workshop Agenda, 2/5/93: V\/'riting the FiveYear Evaluation Report 

Workshop Agenda, 2/26/93: Development of Comniittee Operational Procedures/Guidelines 

Evaluation Form. (Workshop, 2/26/93) 

Narrative: Plan of Operation, Title UI - Strengthening HBCU Program, Institutional Effectiveness, 

Evaluation, and Research Activity, October 1, 1992 - September 30, 1993, April 6, 1992 
Paine College 1992-1993 Fact Book, cover and table of contents 
Resources on assessment, effectiveness and evaluation 



ERIC 



